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Abstract. In this paper we discuss the optimal hquidation over a finite time horizon until the exit 
time. The drift and diffusion terms of the asset price are general functions depending on all variables 
including control and market regime. There is also a local nonlinear transaction cost associated to 
the liquidation. The model deals with both the permanent impact and the temporary impact in a 
regime switching framework. The problem can be solved with the dynamic programming principle. 
The optimal value function is the unique continuous viscosity solution to the HJB equation and can 
be computed with the finite difference method. 
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1 Introduction 

Optimal liquidation has attracted active research in recent years due to the liquidity risk. In a friction- 
less and competitive market an asset can be traded with any amount at any rate without affecting the 
market price of the asset. The optimal liquidation then becomes an optimal stopping problem which 
maximizes the expected liquidation value at the optimal stopping time. In an incomplete market with 
trading constraints on the volume and the rate and with the liquidation impact on the underlying 
asset price, the optimal liquidation is difficult to model and to solve. 

Despite the wide recognition of the importance of the liquidity risk, there is no universal agreement 
on the definition of liquidity. In the academic literature the liquidity is usually defined in terms of 
the bid-ask spread and/or the transaction cost whereas in the practitioner literature the illiquidity 
is often viewed as the inability of buying and selling securities. Black classifies the following four 
major properties of the liquidity: the immediacy of the transaction, the tightness of the spread, the 
resiliency of the market, and the depth of the market. The concept of liquidity can be summarized 
as the ability for traders to execute large trades rapidly at a price close to current market price. The 
liquidity risk refers to the loss stemming from the cost of liquidating a position. 

Due to lack of universal agreement on the definition of liquidity, there are many different forms 
of mathematical characterizations. Apart from commonly used transaction cost and bid-ask spread 
and trading constraints (Cvitanic and Karatzas [4j, Jouini [7], etc.), the other descriptions include, 
for example, that the order of a large investor adversely affects the stock price before being exercised 
(Bank and Baum [T]), that the market has a supply curve that depends on the order size of investors 
(Oetin et al. [3]), that trading can only happen at jump times of a Cox process (Gassiat et al. [B]), 
that the asset price is affected by the permanent and temporary impact of liquidation (Schied and 
Schoneborn |13j), etc. Once the mathematical framework is chosen for the liquidity risk one can then 
study specific problems such as the arbitrage pricing theory, the optimal investment and consumption, 
etc., see |TJ 121 El 111 13 113] and references within. 

This paper studies the optimal liquidation in the presence of liquidity risk. There are several varia- 
tions in the problem formulation in the literature, including finite or infinite time horizon, continuous 
trading or optimal stopping, geometric Brownian motion (GBM) asset price process or Markov mod- 
ulated process, etc. Pemy et al. study the optimal liquidation over an infinite time horizon. The 
stock price follows a GBM process with an extra term that reflects the permanent impact of liquida- 
tion on the asset price and there is no temporary impact. It is a constrained control problem which 
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implicitly assumes that the stock holdings will never be sold out for any admissible trading strate- 
gies. The value function is the unique continuous viscosity solution to the Hamilton- Jacobi-Bellman 
(HJB) equation (two state variables and no time variable). In the continuous time finite state Markov 
chain framework Pemy and Zhang |10j study an optimal stopping problem of liquidation in finite time 
horizon. Pemy et al. |12) discuss the optimal liquidation over an infinite time, similar to that in jll) . 
The main difference is that the asset price follows a GBM process in which the drift and diffusion 
coefRcients are determined by market regimes and the temporary impact of liquidation is reflected 
in the payoff function and there is no permanent impact. The assumptions and the conclusions are 
basically the same as those in [TTj. 

In this paper we discuss the optimal liquidation over a finite time horizon until the exit time. The 
drift and diffusion terms of the asset price are general functions depending on all variables including 
control and market regime. There is also a local nonlinear transaction cost associated to the trading. 
The model deals with both the permanent impact and the temporary impact in a regime switching 
framework. We show that the optimal value function is the unique continuous viscosity solution to 
the HJB equation and can be computed with the finite difference method. Since the time horizon is 
finite the HJB equation also involves the time variable, which makes the discussions and proofs more 
involved than those in [Tl] [12] . Our model setup is in some sense similar to that of Zhu |14j in which 
a finite time cost minimization problem of an insurance company is studied with the surplus being 
modelled by a controlled diffusion process with regime switching. The main difference is, apart from 
studying different problems, that we prove the comparison theorem which leads to the uniqueness 
of the viscosity solution to the HJB equation whereas there is no uniqueness discussion in jl4l . The 
other differences are technical, including the definitions of viscosity solutions (weak and strong forms) 
and the assumptions which guarantee the continuity of the value function. 

The paper is organized as follows. Section 2 formulates the optimal liquidation problem and 
states the main results of the paper. Section 3 gives a numerical example. Section 4 proves that the 
optimal value function is continuous (Theorem p|. Section 5 proves that the value function is the 
viscosity solution to the HJB equation (TheoremTsl) . Section 6 proves the comparison theorem for the 
uniqueness of the viscosity solution (Theorem pi). 



2 Model and Main Results 

Let (r2, J- , P) be a probability space and (Jv)o<r<T be the natural filtration generated by a standard 
Brownian motion process W and a continuous time Markov chain process a, augmented by all P-nuU 
sets. Assume W and a are independent to each other. Assume that the Markov chain has a finite 
state space M = {1, . . . ,m} and is generated by the generator Q = {qij}, where qij > for i,j G M, 
j ^ i and X^JLi ^ij ~ ^ ^'^^^ each i G M. The transitional probability is given by 

pwi+A).,Kt) = .} (1) 

[1 - (7jjA + o(A) itj=i 

for small time interval A > 0. The continuous time Markov chain a(r)o<r<T models the economic 
environment which affects the growth rate and the volatility of the asset price. 

Let r € [t, T] be the time variable, where T is the fixed terminal time and t e [0, T) is the starting 
time. Let S{r)o<r<T denote the stock price and Ar(r)o<r<T the number of shares of stock. Let 
M(r)o<r<T denote the rate of selling the stock, which is a control variable decided by the trader. We 
call u = {u(r)}o<r<T is admissible if it is progressively measurable and u{r) G U for a compact set 
U C [0, oo) for all t < r < T. The stock price S{r) follows a stochastic differential equation with 
regime switching 

dS{r) = fi{r, S{r),u{r),a{r))dr + cr(r, S{r),u{r), a{r))dW{r) (2) 
and the stock holding X{r) follows the dynamics 

dX(r) — —u{r)dr. 

Since the drift and the diffusion terms of S are affected by the trading strategy u there is the permanent 
impact of liquidation on the asset price. Such an impact may be negligible for a small trader (when 
u is small) but can be significant for a large trader (when u is large). We implicitly assume that the 



asset price S{r) is positive for all t < r < T. A sufficient condition that guarantees this is that S 
fohows a geometric Brownian motion process with drift and diffusion coefficients depending on time, 
control and Markov state. We denote by K some generic positive constant which may take different 
values at different places. 

Assumption 1. Functions f = satisfy, for all i, s G [0, T], x,y eR, v £ [0, oo) and £ E M. that 

\f{t,x,v,£)- f{s,y,v,£)\<K{\t-s\ + \x-y\) and \fit,x,v,£)\<K{l + \x\). (3) 

It can be shown, with Assumption [T] that for any admissible control process u gU and any initial 
values {t,s,£) € [0,r) x (0, cx)) x M, there exists a unique solution, denoted by {S^^g{r),t <r< T}, 
to equation ([2]), and that the following inequalities hold: 



E 



sup \Sl,ArW 

re\t,T] 



E[\Sl,/t,)-Sl,^,{t,)\] < Kil + s)\t2~t,\'^^ t,,t,e[t,T] 



E 



sup \Sl,,,,{r) - Sl,^^,{r)\ 



< A'|si-S2|, Si,S2 G (0, oo). 



(4) 
(5) 
(6) 



The proofs of Q , Q and ^ can be found in Mao and Yuan [8] with some minor changes to include 
control processes, see j8j. Theorems. 23, Theorem 3.24 and Lemma 3.3. 

Similarly, {A"^(r),t < r < T} denotes the stock holding and {atj{r),t < r < T} the Markov 
chain process. 

Suppose a trader starts from time t, endowed with initial values {X(t), S{t),a{t)) — {x,s,£) G 
(0, oo) X (0, oo) X M. Define a stopping time 

To = inf{r > t : X^^ir) = 0} A T. 

This is the first time that A"^(r) exits from (0, oo) before or at time T. Since the model is to study the 
liquidation strategy, the trader is only allowed to sell stock without buying back. When the number 
of shares reaches zero before time T the liquidation stops. Otherwise, it stops at time T. 
The expected discounted total payoff associated with a strategy u gU is defined by 



J{t, X, s, £;u) = E 



{u(r)) SlJr)dr + e-^(-''-*)g(A(ro))5(ro) 



where > is a discount rate, (p ^ function measuring the temporary liquidation effect, g a function 
measuring the block liquidation effect, and E the conditional expectation given the information set 
Ft which is equivalent to given X{t) = x, S{t) = s and a{t) = i since the model is Markov. The first 
term is the expected discounted accumulated cash value from the stock liquidation and the second 
term is the expected discounted cash value from the block liquidation at time T for any remaining 
shares of the stock. 

Assumption 2. Functions f = (f>,g are continuous concave increasing onM. and satisfy /(O) = and 
/'(O) ~ 1. Furthermore, function g is continuously differentiable and satisfies, for all x,y that 

Igi^) - 9iy)\ < K\x ~ y\ and \g' (x) - g' {y)\ < K\x ~ y\. 

Note that in a completely liquid market <f>{v) = v and g{x) — x. The objective of the trader is to 
maximize the expected discounted revenue from stock liquidation. The value function is defined by 

V{t, X, s, £) = sup J{t, X, s, £; u). 

For V G U define operators and Q of the value function V by 

dV dV I ^ d'^V 

C"V{t,x,s,£) = -v — {t,x,s,£) + ^i{t,s,v,£) — {t,x,s,£) + -a {t,s,v,£)-^{t,x,s,£), 



and 



QV{t, X, s,£)=Y, <lej (Vit, X, s, j) - V{t, x, s, £)) , 



The HJB equation for the optimal control problem is, for (i, x, s, t) G [0, T) x (0, oo) x (0, oo) x M, 



dV 
~dt 



{t, X, s, I) - sup {C'Vit, X, s, £) + (j){v)s} - QV{t, x, s, £) = 0, 



(7) 



with the boundary condition 



V{t,0,s, 



£) = 



and the terminal condition 



V{T,x,s,£) = g{x)s. 



It is easy to check that the value function is an increasing function with respect to the asset price 
and the stock holding. It also has the following continuity property. 

Theorem 3. Assume Assumptions^ and^ Then the value function V{-, ■,-,£) is continuous on 
[0, T] X [0, oo) X (0, oo) for £eM. 

Proof. See Section 4. 

Since we do not know if the value function V is continuously differentiable and cannot discuss the 
solution to the HJB equation in the classical sense, we need to introduce the concept of the viscosity 
solution to the HJB equation. 

Definition 4. A system of continuous functions V — {V{-,-,-,£)}eeM on [0,T) x (0,oo) x (0, oo) 
is a viscosity subsolution (resp. supers olution) of the HJB equation ([t]) if for each £ € M, (/? S 
Ci'i'2([0, T) X (0, oo) X (0, oo)) and (i, x, s) € [0, T) x (0, oo) x (0, oo) such that V{t, x, s, £) - (p{t, x, s) 
attains its maximum (resp. minimum) at (t,s,x), we have 



Pip{t,x,s) - ^{t,x,s) - snp{C''ip{t,x,s) + (j3{v)s} - QV{t,x,s,£) <0; {resp. > 0). (8) 



The system of continuous functions V is a viscosity solution if it is both a viscosity subsolution and a 
viscosity supersolution. 

The viscosity solution defined in this paper is in the strong sense, i.e., the inequalities need to hold 
at the maximum/minimum point for every market regime. In some literature, e.g. [3 El], the viscosity 
solution is defined in the weak sense that the inequalities only hold at a global maximum/minimum 
point over all regimes. The value function that is a strong viscosity solution is a weak viscosity solution 
as well, but the reverse may not be true. We have the following result for the value function. 

Theorem 5. Assume Assumptions^^ and^ Then the value function V is a viscosity solution to the 
HJB equation ([7|. 

Proof. See Section 5. 

One in general has to use some numerical scheme to find the value function. To ensure the 
numerical solution to the HJB equation is indeed the value function one has to show that the value 
function is the unique viscosity solution to the HJB equation, which can be achieved by the following 
comparison theorem. 

Theorem 6. Assume Assumptio ns [7| and Let U be a viscosity subsolution and V a viscosity 



supersolution to the HJB equation ([7| and satisfy the polynomial growth condition and U{T,x, s,£) < 



V{T, x, s, i) for all {x, s, £) e (0, oo) x (0, oo) x M. Then U <V on [0, T) x (0, oo) x (0, oo) x M. 
Proof. See Section 6. 




3 A Numerical Example 



In this section we apply the finite difference method to find the numerical approximation of the value 
function and the optimal selling strategy. 

Assume that there are only two regimes. Regime 1 represents the strong economy and regime 2 the 
weak economy and assume that the stock price S{r) follows a GBM process with /i(r, s, u, a) — ^{a)s 
and (j{r,s,u,a) = a{a)s. Define variables z ~ logs and t = T — t and a function W{t,x,z,£) = 
V{t,x,s,£). The HJB equation ([t]) becomes 



m{r,x,z,£) 



--—{T,X,Z,e) - sup 



{ 



dW dW 

V^^{T,X,Z,e) + fi{£)—-{T,X,Z,e) 



dx 



dz 

QW{T,x,z,e) = Q, 



(9) 



with the boundary condition W{t, 0, z, £) = and the terminal condition W{Q, x, z, £) — g{x)e^ . 

To approximate the solution to ^ we discretize variables r, x and z with stepsizes Ar, Ax, Az, re- 
spectively. The value of ly at a grid point (t„ , Xi , Zj ) in the regime £ is denoted by W^^ {£) . The deriva- 
tives of W are approximated by 11^^ = (W"+^(€)-II^i"j(^))/Ar, = {W^^^^j{£)-Wl],^^^{£))l{2llx), 



= (VF^-+iW-W^ViW)/(2A^)' and W,, = (W^/^.+i(£)H-M^/^-_i(^)-2W^;^.W)/Az2 
the equation ^ and rearranging the terms, we have 



Discretizing 



W-^'{1) =Ar 



"^+a7-'^"'-^ 



sup 



2Az 2Az2 

U ^ TTl. 

2Aa; 



2Az 

W:^^_,{£)+qu'WUn 



2Az2 



(10) 



where £,£' = 1, 2 and £ ^ £' . Assume that the temporary liquidation impact function is given by 

. 1 



1 



0, 



where a > 0, and the block liquidation impact function is given by 



X, if0<a;<5, 

-0.01a;2 + l.lx - 0.25, if 5 < a; < 15, 

10 + 0.8(x- 10), if 15 < a; < 40, 

-0.0075x2 _,_ ^ 42, _ if 40 < X < 60, 

42 + 0.5(x- 50), ifx>60. 



Functions and g satisfy Assumption [2j In fact, g is constructed as a smooth approximation to a 
function / defined by 

{X, ifO<x<10, 
10 + 0.8(x- 10), iflO<x<50, 
42 + 0.5(x- 50), ifx>50. 

Function / captures the block liquidation effect at time T but is not differcntiable at x = 10 and 50 
and does not satisfy Assumption |2] 

Data used for numerical tests are a = 0.005, /3 = 0.01, = {0.3,-0.1}, {cr(l), cr(2)} = 

{0.2, 0.4}, qi2 = 0.5, q2i = l,v = [0, 100], t e [0, 1], x e [0, 100], s £ [e-\e% 

Figure [T] demonstrates the relationship between the optimal selling strategy and the stock holding. 
It is clear that the more shares one holds, the sooner and the more one wants to sell to avoid the 
potential large transaction cost during the whole period. The market regime determines at what level 
of stock holding one should start to sell. In a rising market (regime 1) the trader is willing to keep 
the stock for a longer period in the hope for a higher price, which results in a lower optimal selling 
rate, whereas in a falling market (regime 2) the trader wants to liquidate the stock quickly to avoid a 




Figure 1: The optimal control at time t — against stock holding x. The solid line is for regime 1 
and the dashed line for regime 2. 



lower price. This is consistent with the general market phenomenon. The optimal trading strategy is 
independent of initial asset price in the numerical test, which is not surprising as the asset price follows 
a GBM process and depends on the initial asset price linearly. In general, the optimal trading strategy 
should also depend on the asset price. The particular shape of the curve in Figure [T] is determined 
by the tradeoff between function (/) that captures the liquidity effect from 'flow' trading and function 
g that reflects the transaction cost for the block liquidation at the terminal time. Note that if there 
is no temporary pricing impact on liquidation, i.e., (I>{v) ~ v, then the optimal liquidation strategy is 
a "bang-bang" control with either no trading t; = or selling at maximum rate v = 100 due to the 
linear dependence of control v in the Hamiltonian function. 



4 Proof of Theorem [3] 



We first convert the original control problem into a problem without terminal bequest function. Since 
function g is continuously differentiable, we can apply Dynkin's formula to e^'^'^°~*^g{Xf^(To))S^^ ^{tq) 
and rewrite the total payoff J as 



J(i, X, s, £; u) — g{x)s + E 



Lir,Xl,ir),Sl,^,ir),u{r),atAr))dr 



where 



L(r, x, s, a) = e [(l){v)s — /3g{x)s — vg' {x)s + ^{r, s,v,a)g{x)] . 

Define a new value function by 



V{t,x,s,£) = sup E 
ueu 



L{r,Xrjr),Sl,/r),u{r),atAr))dr 



Since V{t,x,s,i) = V{t,x, s,i) + g{x)s, we know V{t, x, s, i) is continuous as long as V{t,x,s,£) is 
continuous. From now on in this section we work on the value function V. 

To prove the continuity of V we need to define some perturbed problems and show their correspond- 
ing value functions are continuous and converge quasi-uniformly to V, which establishes Theorem [s] 

For < e < 1 define the stopping time 

T, = m{{r>t:Xrjr) = -e}AT, 

which is the first time X"^(r) exits from (— e, oo). A control process u = {u{r)}o<r<T is admissible 
if it is progressively measurable and u{r) e Uf{X^^^{r)), where Ue{x) = U ii x > and Ue{x) = U, a. 

compact subset of C/ in (0, oo), if a; < 0. The key here is to rule out zero from the compact set U after 
X{r) reaches zero. The admissible control set is the collection of all admissible controls, denoted by 
14^. Note that when we only look at the control process before rg, the two admissible control sets, 14 
and Ue, are the same. 



To simplify the notation denote by 



Ll.^sA^) L (r, Xl,{r), Sl,^,{r),u{r),atAr)) ■ 

Since [/ is a compact set in [0,oo), say [0,-/V], we know that Xf^{r) e [a; — NT,x\ ioi t < r < T, 
which impUes that \g{Xf^^{r))\ and \g' {X^ ,^{r))\ are bounded by some constant depending on x 
due to continuity of g and g' . Assumptions [I] and [2] imply that, for t < r < T, 

\L'L,sA^)\<K4l + Sl,Ar)) (11) 

and 



\Ll.,,s,A^) - Ll,.,s,A^)\ < \Sls,A^) - Sls,A^)\ +K{1 + Sl^^A^)) \x, - X2\ (12) 
for some constant K^^ depending on xi. 

Remark 7. In the proof we need to estimate ^(r)| several times for different x. One case is 

that X = —e for < e < 1. Then X^^{r) G [— 1 — NT, 0] and constant Kx can be replaced by a generic 
constant K independent of x. The other case is that x is within a distance d of another point Xi. 
Then X^^{r) € [xi — d — NT, xi + d] and constant can be written as K^^ depending on xi for all 
such X. 



For e S (0, 1) define a perturbed value function by 



V'{t,x,s,£) = sup E 
For p > define an auxiliary function 



n;r(r)-exp --(X,^Jr) + .) 



where x = max(0, —x). Clearly, we have r('^'"(r) < 1 and, by the definition of the stopping time r^, 
r('^'"(r) = 1 for r E [t, t^]. The auxiliary value function V^ ^ is defined by 



V'^it, X, s, £) = sup r'P{t, X, s, i; u) E 



Tli'^{r)LlAr)dT 



From (111 and Q we have that 



\V'^f'{t,x,s,i)\<Kx{l + s). (13) 

Lemma 8. V'^'''{t, x, s, £) converges to V{t, x, s, t) quasi-uniformly as p — > and e — > 0. 

Proof. Step 1. Fix a point {t,x,s) <E [0,T) x {— e} x (0,oo). Since Xl')^{t) = — e we have = t and 
for r > i the admissible control u{r) is in a compact set Ue{x) :— [Nq,N] C U with A'o > 0, which 
implies that X"_j(r) < — e and 



exp — (r - t)^ < rl'P:^{r) = cxp (^-- ^ u{s)ds^ < exp 



No 



(r-t) 



(14) 



and limp^o r(''^'"(r) — 0. (14), (11) and (j4j) imply that, also noting Remarkjlj 



{t,-e,s,i;u) <K e 



l + E[Sl,^,{r)\)dr<K{l + s)^^{\- 



'"a rp 

e p 



Similarly, we have 



r-P (t, -e, s, £; u) > -K{1 + s)^ ( 1 - g- p 



Combining the above two inequalities and taking the suprcmum, we have 



P 



-K{1 + s)j^[l- e-f ^) < V^'P {t, -e, s,£) < K{1 + (l - e"^^) 



Applying the dynamic programming principle, for (t, x, s, £) G [0, T) x [0, co) x (0, oo) x M, we have 



V''P{t,x,sJ) = sup E 



< sup E 



1 — e p 



<V\t,x,s,t)+K{l + s)-^[l- 



"0 y 



Similarly, we have 



'(i, X, s, > V\t, X, s, ~ K{1 + s)^{l- e-^^) 



The above two inequalities imply that ^^'''(t, x, s, ^) converges to V^^t^x, s,t) quasi-uniformly as 
p — >■ 0, independent of e. 

Step 2. By the definition of the perturbed value function, the Cauchy-Schwartz inequality, (11) 
and Q, we have 



V^{t,x,s,i) = sup E 



<V{t,x,sJ) + sup E 

u&A, 



l{ro<r<rji"x,s,£('^)c^'' 



<V{t,x,sJ)+ sup ^/E[t^ - To] 



\ 



<F(t,2;,s,^)+if,(l + s) 



1/2 



for some constant i^T^, depending on x. Similarly, we have 



Vit, X, s, I) > Vit, X, s, £) -K^Xl + s) 



1/2 



As £ — 0, V'^{t,x,s,£) converges to V{t,x,s,£) quasi-uniformly. Combining the results of Steps 1 
and 2, we conclude that V'^'P{t, x, s, £) converges to V{t, x, s, £) quasi-uniformly as p — >■ and e -> 0. □ 

Lemma 9. •, •, ^) is continuous on [0,T] x [0,oo) x (0, oo) for £ e M and arbitrary constants 

e > and p > 0. 

Proof. Step 1. Let (xi,si), (a;2,S2) G [0, oo) x (0, oo) satisfying \x2 — xi\ < 1 and |s2 — si| < 1 and 
t S [0,T] and £ & Consider the auxiliary value functions V'^'''{t,xi, si,£) and V'^'''{t,X2,S2,£)- 



Since 



< |a — 6| for any a, & > 0, we have 



1 



1 



~ n.r M < - m..A^) + (km + e)- < - ki - x^i 



(15) 



By the definition of V^'P and the relation | sup A — supi?| < sup \ A — B\ we have 

V'-Pit,X,,Su£)~V'^'{t,X2,S2j) 
T 



< sup E 

u<£U, 



< sup E 



rli\''ir)L-Jr) - Tt:^,^y{r)Ll,Jr) dr 



1, 



<K^^\S1 ^S2\+K{l+S2) \xi -X2\ + -\xi- X2\K^^ (1 + S2) 

P 

<K^^^s^{\xl - X2\ -f |si - S2I), 



(16) 



where -ftT^jo, is some constant depending on xi and si. In the second last inequahty we have used 
(12), |4|, (11), (15) and Remark]?] This shows that the auxihary value function V'^'P{t,x,s,£) is 
continuous in (x, s), uniformly in t. 

Step 2. We prove that the auxiliary value function V'^''' is continuous in t. Let < ti < t2 < T 
and {x, s, i) S [0, oo) x (0, oo) x M. By the dynamic programming principle, for any 6 > 0, there exists 
an admissible control us G Uc such that 

V''''{ti,x,s,£) - 5 

<e\ f\lf::^r)L^l^^^/r)dr + e-^^'^-'^^V^^^^ 

<V''Pih,x,sJ). 

Rearranging the above inequalities, we have 



v''''{t,,x,s,e)-v'^p{t2,x,s,e) 



< 



E 



V''P{t2,x,s,e) 





/•t2 










<E 






dr 


+ E 





E 



e-^^'^-'^^V'^P{h,Xr,%{h),Sl%/h),e)-V^^P{h,x,s,£) 

(TT| and ^ imply that 
(13) and Remark [T] imply that 



E 



for some constant K^^s depending on x and s. Noting that the term inside the expectation of I3 is 
zero when at^/{t2) — £, using Cauchy-Schwartz inequality and combining the above inequality, we 
have 



Using (16) and ([s]), we have 



h <E 



<K,AE [\X^,Ut2) - x\] + E Sl[^,{t2)-s ) + E V''P{t2,x,sJ) 



<K.,,s({t2-h) + {t2~hf'^ + 



-/3(t2-tl) 



1 



for some constant Kx^s depending on x, s. The above estimates for /i,/2,/3 show that they all tend 
to as ^2 — ti tends to 0, independent of 5 and control ug but dependent on x and s. Therefore, 



V''P{tux,s,l)-V'^''{t2,x,s,£) 



(5 as ^2 - ^1 ^ 0. 



The arbitrariness of 5 confirms that V^'^lt, x, s, £) is continuous in t. 

Combining the results of Steps 1 and 2, we conclude that V'^'''{-, •, ■,£) is continuous in {t,x,s) for 
each eeM. □ 



By Lemmasj8| and |9j the auxiliary value function V'^'''{t, x, s, £) converges quasi-uniformly to the 
value function V{t, x, s,£) a,s e ^ and p — and V'^'P{t, x, s, £) is continuous in {t, x, s), which shows 
that V{t, X, s, £) is continuous on [0, T] x [0, 00) x (0, 00) for each £ S M. We have proved Theoremji] 



5 Proof of Theorem [5] 



We first show that is a viscosity supersolution. 

Theorem 10. Given Assumption^ the value function V = {V{t,x,s,£)}^^-f^ is a viscosity superso- 
lution of the HJB equation ([?]) . 

Proof Let i eM, {t,x,s) g [0,r) x (0, oo) x (0,oo). Let the test function ip{t,x,s) € C'^''^''^{[0,T) x 
(0, oo) X (0, oo)) such that V{t, x, s, £) — ip{t, x, s) attains its minimum at (i, x, s) and, without loss of 
generality, V{t, x, s, £) — (p{i, x, s) = 0. Choose a constant control u{t) = w G [/ for t e [0, tq]. Let the 
state variables X and S start from time t with initial values x and s. 

Define fi as the first jump time of the regime af^^ (•). Without loss of generality, assume that rj is 
small enough such that 5^(5, s) C (0, oo) x (0, oo). Define T2 by 

h := inf {r>i: (X«^(r), Sl,/r)) ^ B,, {x, s)] . 

For h < T — t, define the stopping time t (i + ft,) A fi A f2. Note that t < tq. By dynamic 
programming principle. 



V{t,x,s,i) > E 
Define 



-^^^~'')<j>{u{r))Sl,^,{r)dr + e-P^^~'W{T,Xl,^^^ 



ip{t, X, s, i) 



(p{t, X, s) if z = i, 
V{t,x,s,i) Mi^i. 



(17) 



(18) 



Applying Dynkin's formula at point (t,a;, s,^), also noting 'ip{t, x, s, £) — (p{t,x,s), we have 
E [e-«--*^V {T,Xl,{T),Sl,^,{T),au{T))_ 
=^{t,x,s) + E [{-P)e-^^^-^^{r,Xl,{r),Sl,^,{r)) 

which implies, from the choice of (i, x, s, £) and the definition of ■(/', that 
e-^(--*^V(r,X«,(r),5,",-,(T),ao(r)) 



(19) 



E 



>V{t,x,sJ) + E 



(-P)e-^(r-t)^^^^XlJr),Sl,/r)) 



-'"^'-'^ (I + V {r. XUr), Sl,,{r)) + QV (r, X^r), Sl,/r),i) | dr 



(20) 



Substitute (20) into (17 1 and divide both sides by — /i we get 

d 



<E 



1 



e-^f'--*) ( (r, X«,(r), Sf-^Ar)) - ( ^ + ) ^ (r, X|,(r), S'l^jXr)) 



(/.(u(r)) Sl,^,{r) - QV (r, X|^,(r), ,(r), dr 



<E 



t+h 



+ K 



cl^{u{r))Sl,j{r)^ - QV{r,Xl,{r),Sl,/r),i)'^dr 

i^[(nAf.-t-)|nAf.<t-+ft]^^.^^.^^-^^^ 



Tl AT2 > t + h 



P [fi A fs > t + ft] 



ft 



(21) 



for some constant i^T, due to continuity of the function on the left hand side of ([s]) and the boundedness 
of state variable on the time interval [0, fi A T2]- 



By definition of fi, we liave 

P[fi <t + h]^l- P [atjir) ^ e, r e {t,t + h]] = 1 ~ e«"''. 
So as ft. — !• 0, P [fi < < + h] goes to zero. By Cliebysliev's inequality, we liave 



P[f2<t + h] =P 



sup {\Xl,ir) - x\' + \Sl,/r) - sf > v'} 

re[t,t+h] ■ 



< 









2 








2" 


E 


SUPr6[t,t+/i] 


X^Jr)-x 




+ E 









(22) 



Since eacfi term on the numerator of ( 22 ) converges to zero as ft. and lim/i^o P [t2 ^ t + h] = 0, 
we have 



lim F [fi A fa < < + ft] < lim [P [h < t + h] + P [t2 < t + ft]) = 0. 

/i— i-O h—^0 



(23) 



Let ft — > in (21 ). By the mean value theorem and the dominated convergence theorem, we have 

d 



pip il X, s) - (^^ + -C" j (t, 5, s) - <j){v)s - QV (i, i, s, £) > 0. 

Since u{r) = w e J7 is chosen arbitrarily, we take the supremum over U and get 

- _ _ d - _ _ -__ _ -__ 

fiip {t, x,s) - —if {t, X, s) - sup {C^ip (t, X, s) + (f>{v)s} - QV {t, x, s, I) > 0. 

Ot ^jfZjJ 

Therefore, V is a viscosity supersolution of the HJB equation ([T]). 
For £ gM, define the Hamiltonian function H by 

Hit, x,s,p, q, M,£) :— sup < —vp + fi{t,s,v, £)q + -a'^{t, s,v,£)M + (j){v)s 
veu I ^ 



□ 



(24) 



Lemma 11. For all i G M, the Hamiltonian Hit^x, s,p, q, M, £) is continuous in [t, x, s,p, q, M) G 
[0,r) X (0,oo) X (0,oo) X E X K X M. 

Proof. Let the point {t, i, s,p, q, M) G [0, T) x (0, oo) x (0, oo) x M? and Bj^{i, i, s,p, q, M) the ball with 
the center (t,x,s,p,q,AI) and the radius 77, a small constant. By the definition of the Hamiltonian 
function, for an arbitrary given 6 > 0, there exists a v & U such that 

1 



n {t,x,s,p,q,M,i) - (5 < -~vp + fi{t,s,v,i)q+ -cr^ {t,s,v,e)M + (j){v)s. 

For any point {f , x' , s' ,p' , q' , M') G B,^{t, x, s,p, q, M) we also have 

H {f, x', s',p', q', M', £) > -tip' + fi {t', s', V, £) q' + {f , s', v, £) M' + 4, {v) s' . 

Subtracting ( [26| from ([25|, we have 

n{t,x,s,p, q,MJ)-H{t\x',s\p',q',M',£) ~S 

+ ^cr^{i,s,v,i) \M-M'\ + ^ |M + ?7| \a^{t,s,v,£) - cr^ {t' , s' , v , £)\ + \(f>{v)\ \s-s'\ . 

Taking the limit inferior and then letting S tend to zero in ([27]) we get 

■Hit,x,s,p,q,M,£) < liminf Hit', x' , s' ,p' ,q' , M' ,£). 

(t' ,x' ,s' ,p' ,q' ,M') 
-^{i,x,S.p,q,M) 



(25) 
(26) 

(27) 
(28) 



Similarly, we can show, using the uniform continuity of /i(-, •, •, £) and cr(-, •, ■,£) and the bounded- 
ness of the control set U, that 



lim sup n{t',x',s\p',q',M',£) <'H{t,x,s,p,q,M,£). 

{t' ,x' ,s' ,p' ,q' ,AI') 
-^{i,x,s,p,q,M) 



(29) 



(28) and (29) imply that the Hamiltonian Hit, x, s,p, q, M, £) is continuous in {t,x, s,p,q, M). 



□ 



For ip € C^'^''^ Theorem [3] and Lemma 11 imply that the mapping 



{t, X, s) I— >■ /3</'(ij X, s) 



^{t, X, s) - sup {C^ifit, X, s) + 0(t;)s} - QVit, x, s, £) (30) 



is continuous. 



Theorem 12. For each I e M, the value function V — {V{t,x,s,l)}tizyi is a viscosity subsolution of 
the HJB equation Q. 

Proof. Assume, for contradiction, that V is not a viscosity subsolution. Then there exists £ e M, 
(t, X, s) £ [0, T) X (0, 00) X (0, 00) and a test function (p{t, x, s) e €^^'^'^{[0, T) x (0, 00) x (0, 00)) such 
that 

!iip{i,x,s) - ^{t,x,s) - sup{£"'^(i,x,s) + (/)(u)s} - QV{t,x,s,i) > 0, (31) 

ot 1,^1/ 

where V{t, x, s, £) — ip(t, x, s) attains its maximum at {t, x, s). Without loss of generality, assume that 
V{i, X, s, £) — ip{i, X, s) = 0. 



By the continuity of the mapping in (30 1, for (5 > 0, there exists 77 > such that 

jSip (t, X, s) — ^ (i, x, s) — sup {C"ip (t, X, s) + 0(u)s} — QV{t, x, s,£) > S 
ot 



(32) 



for all {t, X, s) £ B,f{t, x, s). Let rj be small enough such that B,^{t, x, s) C [0, T) x (0, 00) x (0, 00). 

Let ft- > be small enough such that {t, i + h) d [0, T). By dynamic programming principle, there 
exists a control process u £ U such that 



V{t,x,s,£) - -h<E 



e-P^^-^cj> {u{r)) Sl,/r)dr + e~P^^-^V (r, X«,(r), 5t-,,(r), «M(r)) 



(33) 



where r > t is any stopping time. Let fi be the first jump time of «(/(■) a-nd define the exit time 

f3 inf {r>t: (r, Xl^{r) , Sf-, ^,{r)) ^ B^{1 x, s)] . 
Let r := (f + /i) A fi A and define a function V'(i, x, s, i) as in ( 18 ). We have 



and 



(r, X|^,(r), 5f ,(r), a,-,(r)) > F (r, X«,(r), ,(r), af,,(T)) 
QV' (r, X«,(r), ,(r), £) < QV (r, X,",(r), Sl,^,{r),£) . 



So equation ( 33 ) turns into 
^ {i, X, -s)-^-h<E\ r {u(r)) Sl,^,{r)dr + e'^^^-'^^ (r, X^;,(r), ^,",_,(r), af,,(T)) 



(34) 

(35) 



Combining (19 1, (34) and (35), we divide both sides of the equation by h, 

d 



1 



dt 



£u{r) 



V (r, Xl,{r),Sl,/r)) - (u(r)) ,(r) 



(r,X,"^(r),5||,-,(r),^) 



Substituting i32l into ( 36 ) , we have 



5 5,^ 
0>-^ + -^E[r^tl. 



(36) 



(37) 



By ( 23 ) , we have 



l>^E[T-t\>^E[h\fiAT3>t + h]P[TiAf2>t + h]^P[TiAf3>t + h]^l 



as ft — >■ 0, which implies that 



lim -E\T-i\^l. 
h^o h 



Letting ft — > in (37), we get (5/2 < 0, a contradiction. The inequality in (31) therefore holds, 



which completes the proof. 



□ 



Since the value function V is both a viscosity subsolution and a viscosity supersolution, we conclude 
that it is a viscosity solution of the HJB equation ([7| . We have proved Theorem [5j 



6 Proof of Theorem [6] 

In this section vectors {t,x,s) and {r,y^v) and their specific values such as {i,x,s) appear many 
times. To simphfy the expressions we denote by x = {t,x,s) and y — {r,y,v). Their specific values 
are defined similarly, for example, x = (t, x, s). 

To prove the uniqueness, we need an alternative definition of viscosity solution in terms of superjets 
and subjets. The second-order superjet of an upper-semicontinuous function U at a point x G E := 
[0, T) X (0, cx)) X (0, oo) , denoted by 'P'^^+U{±), is defined as a set of elements {b,p,q, M) eRxRxRxR 
such that 

C/(x) < C/(x) + (b,p, • (x - x) + ^M{s - s)2 + e(x - x), (38) 
where e(x — x) — o{\t — t\ + \x — x\ + |s — s|^) is a higher order error term. The limiting superjet 

2 -j- 

V ' U{x) is the set of elements {b,p,q,M) G M.'^ for which there exists a sequence (x^) in S and 
(6„Pe,ge,Me) G 7'2.+t7(x,) such that (x„[/(x,),6,,p„g„M,) ^ (x, t/(x), g, Af). 

The second-order subjet of a lower-semicontinuous function at a point x G E, denoted by 



2 

■p^^^y(x), is defined as in (38) with a greater than or equal (>) inequality. The set V ' F(x) is 



defined similarly. 

Note that since x is a state variable superjets and subjects should normally also have second order 
terms with respect to x. However, since the HJB equation ([7| only involves the first order derivative 
of the value function with respect to x, the second order expansion in x is not needed. 

Assume that U is upper-semicontinuous and (p G C^'^'^(S). Then x G S is a maximum point 
of [/ - if and only if {D^ip{x.) , D^^x)) G V^^+U{±), where D^ip{x.) = {Dtip{^), D^(p{±), Ds^{±)). 
Similar conclusion holds for the minimum point and the subjet. 

Lemma 13. (J^ Theorem 8.3]) An m-tuple V = {V{-^ •, ■,^)}£gm of continuous functions on is a 
viscosity subsolution ( resp. supersolution ) of the HJB equation ([7| if and only if for x G E such that 
{b,p,q,M) eV^'^V{x,e) (resp. eV^'^V{x,£)) for any fixed £ G M, we have 

f3V{x,£) -b^ni:>i,p,q,M,£) - QV{x,e) <0 {resp. > 0), 



where 'H{x,p,q,M,£) is the Hamiltonian define in (24). The m-tuple V is a viscosity solution if it is 



both a viscosity subsolution and a viscosity supersolution. 

The uniform polynomial growth condition for U and V implies that there exists a constant p > I 
such that, for each £ G M 

sup 1 ^ IP Imp < 

[0,T]x(0,oo)x(0,oo) J- + Fl + |S| 

Define functions 9{x,s) (1 + \x\^^ + \s\^^) and K{t,x,s) := e^^^O{x,s) for 7 > 0. Due to the 
linear growth condition ([s]) and the boundedness of set [/, there exists a positive constant c such that, 
for all £ G M, 

/3k - — - sup {Ck} 

01 ^(=u 

Ok ( 1 

=I3k — — sup < —vDxK + ii{t, s, V, £)DsH + s, v^ £)D1k + Qk 

'yt veil L 2 



{13 + -f)9 - sup <{ -vD^e + n{t, s, V, £)DsO + ^<J^{t, s, v, £)Dle + QO 



veu 

>e-^\^ + j-c)0, 

which is nonnegative as long as we choose the constant 7 large enough such that (/? -f 7 — c) > 0. 
Therefore, for any e > 0, V'^{x,£) := V{x,£) + eK(x) is a supersolution to the HJB equation Q. To 
check this, let <p(x, be the test function for V'^{x,£). So <f{x,£) — €k{x) is the test function for the 
supersolution V{x,£). We have 

/3(p-|^-sup {C"^ + cj,{v)s} 
at ijgj/ 

>I3{lp ~ en) ^ ^{^ — — sup {C'"{'f — en) + 0(u)s} + e ( /3k — ^ — sup {C" k] 
at ^,g(7 \ at -ueu 

>0. 



By the polynomial growth condition of U, V and the definition of k, we have 

lim sup {U ~V'){x,e) ^~oo 

for all e > 0. We can assume that the maximum of ([/ — V^)(x, £) over ^ e M and x € [0, T] x (0, oo) x 
(0, oo) is attained (up to a penalization) at £ G M and x e Si [0, T] x Oi x O2 for some compact 
set Oi C (0,00) and O2 C (0,oo). Let A4 denote this maximum. 

Suppose, for contradiction, that there exists £ e M and x e E such that U{x,£) > V{x,£). We 
have 

M:^max sup (U-V)(yi,i)= max (U - V)(x,i) > 0. (39) 
»eM [o,T]x(o,oo)2 ieM.xeSi 

For any e > 0, define a function 5''^ by 

vl/^(x, y, e) := [/(x, £) - V{y, £) - 7A^(x, y), 

where tp'^ is defined by 

V^^(x,y):=i|x-y^ (40) 

For each £ G M, ^'^(•, •,^) is continuous. Hence its maximum, denoted by over the compact 
set El X El can be attained at (x^,y^). Assume that the maximum M'^ := max^gMiA^^ is attained 
at e M and (x|,,y|,)- We have 

M<M' =vi/^(x^.,y|.,£') < U{x\.,t) - V{yl._,t). (41) 

As e — > 0, the bounded sequence (x|5,y^,) converges, up to a subsequence, to a limit (x,y) G 
El X El. By assumption, M is finite. For each £ G M, the sequence (x^,y|) converges, up to a 
subsequence, to its limit, respectively. Therefore, for e small enough, £^ = I for ^ G M. 

Since {?/(•, ^)}£gm and {V^(-,£)}£gM are continuous and M is a finite set, U{yi\,,l'^) — V^(y£,,^'^) is 
bounded for all e > 0. From ( [4l] ), -0*^ (x^^ , y^^ ) is also bounded, which implies that 

lim(xi,yi) = (x,x), Xxm = M = (U -V)(x,l), lim#(xi,yL) = 0. (42) 

By applying Ishii's Lemma (see Lemma 4.4.6, Remark 4.4.9]) to function ^1''^ at its maximum 
point (x^, , y\,) with £ = we can find M% TV" G M such that 

i(x^.-y|.),Ar) GP''+c/(x^,,n, (^-{A^-y\^),N^\^v'''y{y\^.n 



and, for any c, c? G M, 



Denote by 



c^M' -SN' <'U^c-df. (43) 



e \€ e e 

Since ?7 is a viscosity subsolution and V a supersolution, by the definition of viscosity solutions in 
terms of superjets and subjets, we have 

/?C/(x|. ,n-vl- QU{xj, ,n-n (x|. , ry^, ry^, M% r ) < (44) 
pViyl. .n-v{- QV{y\. ,n-n {y\. , , 77^ , 7V^ r ) > 0, (45) 

where the Hamiltonian % is defined in ( p4| . By the definition of operator Q we have 

Q{U{x\._,n-V{y%,n) 

^ 9^'. [(C/(x|.,j) - V{yl.,j)) - (C/(x|.,r) - F(y^.,r))] 

= qe^j [«'^(x^. , , j) - (x|. , y^. , n] 

<0. (46) 



The last hue is from the fact that ^"^(x^j , y^^ , is the maximum of 5''^(x,y,£) over £ S M and 
(x,y) e Si X Sj. 



Subtracting ( 45 1 from (W4| and rearranging, also noting ( 46 1 , we have 



P (C/(x^. , n - V{y\. , n) < n (x^. ,r,^„7f„M\n-H {yl. , ry^, 77^, iV% r ) . (47) 
By the definition of the Hamiltonian function, for any (5 > 0, there exists a, € U such that 

H (x|. ,7^lr,l,M\t)^d< -u^^ + M , si. , v', + ^a^ {tl. , s\. , v\t) + <i>{v')s\. . (48) 

We also have 



(49) 



Subtracting ( 49 1 from ( 48 ) , we get 



<vl[f^ (^^.,s^<,^;^r)-/i(r•|.,^;,^,^;^r)] 



(50) 



Here we have used MSl 



By Assumption nTon /i and cr, (42) and the boundedness of 4'{v ), the right side of (50) tends to 



as e — >■ 0. Since i5 > is chosen arbitrarily, we have 



Hm sup {n (x|. , 77^ , , M% r ) (y|. , t?^ , 77^ , iV% ^ ) } < 0. 

e-s-O 



(51) 



Combining (42), (47) and (51), we have 

/3([/(x,l) -^(x,!)) <0, 
which contradicts ( 39 ) . Therefore U <V on [0, T) x (0, 00) x (0, 00) x M. We have proved Theorem|6] 
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